This is an automated cherry-pick of #10764

What problem does this PR solve?

Issue Number: ref #10763

Problem Summary:

• We need continuous and low-noise observability for remote object-store usage per TiFlash store.
• Existing S3GC visibility is insufficient to quickly compare expected valid data size vs real object-store footprint.
• We also need this summary collection to be configurable and safe for production overhead.

What is changed and how it works?

disagg: add configurable owner-only S3 storage summary and per-store usage metrics

• Add periodic S3 storage summary in S3GCManagerService:

  • Runs only on S3GC owner.
  • Interval is configurable via summary_interval_seconds.
  • If summary_interval_seconds <= 0, the periodic task is not registered.
  • Warn when interval is too small (< 12h) to avoid high S3 listing overhead.
  • Disabled by default until we ensure the cost is acceptable.
  • If we need to check this metrics, we can trigger the HTTP API to get a metrics and inspect on it. http://${TIFLASH_IP}:${TIFLASH_STATUS_PORT}/tiflash/remote/info https://github.com/pingcap/tiflash/blob/master/docs/tiflash_http_api.md#fetch-the-remote-storage-summary-from-tiflash-write-node

• Add graceful shutdown behavior during summary scan:

  • getStoreStorageSummary checks shutdown_called in listPrefix loop and exits early.

• Add per-store summary metrics:

  • New gauge family:
    • tiflash_storage_s3_store_summary_bytes{store_id, type}
    • type in {data_file_bytes, dt_file_bytes}
  • Metrics are updated after each getStoreStorageSummary completion.

• Wire configuration from settings to S3GC config:

  • New setting:
    • profiles.default.remote_summary_interval_seconds (default 0, disabled)
    • Related docs: tiflash: Add S3 GC related configs docs-cn#21455
  • Set in TMTContext and passed into S3GCManagerService.

Check List

Tests

• Unit test
• Integration test
• Manual test (add detailed scripts or steps below)

# Run chbenchmark workload and check the metrics of `prelock_keys` and OSS usage
tiup bench ch --host 10.2.12.81 -P 8081 --warehouses 8000 run -D chbenchmark8k -T 50 -t 0 --time 30m --ignore-error --queries q1
# Before the fix, from 23:29 to 00:00, the number of prelock_keys in memory would accumulate and increase with the write load; after the fix, from 02:00 to 02:30, there was no longer any persistent residue of prelock_keys in memory.
# Also can check the new added grafana panel "Remote Store Summary (Disagg arch)"

Side effects

• Performance regression: Consumes more CPU
• Performance regression: Consumes more Memory
• Breaking backward compatibility

Documentation

• Affects user behaviors
• Contains syntax changes
• Contains variable changes
• Contains experimental features
• Changes MySQL compatibility

Release note

Add configurable owner-only periodic S3 storage summary and per-store summary metrics (`tiflash_storage_s3_store_summary_bytes`) for TiFlash disaggregated S3GC observability.

Summary by CodeRabbit

• New Features

  • Added per-store S3 storage summary metrics for data and index file sizes.
  • Added configurable periodic S3 summary collection (default 24h; ≤0 disables).
  • New Grafana panels: Remote Store Summary, Local Lock Manager status, and Local Lock Manager QPS.

• Documentation

  • Updated dashboard layout and panel organization for improved metric visibility.